Neural networks including deep neural networks may be used for machine learning and perceptual and cognitive systems. Such neural networks may be used in a variety of implementations such as speech recognition systems. For example, neural networks may include interconnected layers of neurons or nodes. Input values for each layer include inputs to the system (e.g., at the input layer) or outputs from a previous layer in the neural network. Output values from the output layer of the neural network may include output values, distance values, or classification values, or the like such that the input to the neural network may be classified via the neural network and/or additional processing. Such neural network processing and classifications may be used to perform classifications or other tasks that may be difficult or impossible to perform with more standard rule-based processing systems.
As discussed, such neural networks may be implemented in automatic speech recognition (ASR) systems and, in some implementations, they may be the most important component of such systems. A problem with current neural networks in real-time applications is the large computational effort needed to evaluate the neural network. To address this problem, some current implementations offload neural network computations from a central processing unit (CPU) of a device to a graphics processing unit (GPU) of the device. However, such offloading may cause conflicts with other GPU-intensive applications such as games being run on the device or the device's camera or the like. Furthermore, intensive use of the device's GPU may increase power usage and thereby limit battery life for mobile devices. In other implementations, single instruction, multiple data (SIMD) platforms and/or optimizations such as batched lazy evaluation models (which may delay calculations until needed) may be used. However, such implementations may have reduced classification accuracy.
As such, existing techniques do not provide real-time, efficient, and accurate neural network implementations. Such problems may become critical as the desire to utilize classifications via neural networks such as in speech recognition becomes more widespread.